Ollama with Docker Compose

#AI #docker #docker-compose #container

Get the Ollamaapp running on your local machine, but containerized by Docker for better flexibility in deployment and portability

I'm running this on my MacBook Pro M1 Laptop

Create a Directory

mdkir ~/docker/ollama-container
cd ~/docker/ollama-container

CPU Only (no GPU)

docker run -d -v ./ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Installing Models

docker exec -it ollama ollama run codellama
docker exec -it ollama ollama run codellama:7b-instruct

Create an Alias

If using a Linux machine with Bash. This will create a command line shortcut for a more convenient execution of this container

# setup the alias
alias ollama="docker exec -it ollama ollama"

# using the alias
ollama run codellama

Exit Chatbot

/bye

Code Editor Extension

currently trying to get this hooked up with rjmacarthy/twinny VS Code extension, but having some trouble. Currently following this issueto find out how to use it. For now I just installed the native Ollama app to my desktop (which pretty much defeats the purpos of this writeup for me lol)

Credits